For many types of images, it is useful to capture image data using multiple devices. Such devices include, for example, image capture devices (cameras and the like) and/or lighting devices (flashes and the like). Such image data may be captured sequentially or concurrently. Capture of multiple images using multiple devices allows for many useful processing techniques, including, for example, generation of composite imagery wherein the multiple images are processed and combined, as well as generation of interactive imagery that allows a user to provide input and thereby affect the way the image is presented.
One application that is of particular use is relighting, wherein the lighting characteristics of an image can be controlled, changed, and/or made interactive. In general, relighting effects rely on the capture of multiple images, wherein each image has different lighting characteristics. The resulting set of images can be processed, assembled, related to one another, and/or otherwise manipulated in various ways.
Existing relighting techniques include interactive approaches wherein images with different lighting characteristics are combined differently depending on user input. For example, in response to a user moving a slider or other control, different images can be combined with different relative weights, so as to change the lighting characteristics of the resultant combined image. Various types of filters can be applied to broaden or narrow the lighting characteristics for each component image, and thereby attain sharper or flatter lighting in the resultant image. Additional techniques are described, for example, in Ng et al., “All-Frequency Shadows Using Non-linear Wavelet Lighting Approximation”, ACM Transactions on Graphics, July 2003; and Ng et al., “Triple Product Wavelet Integrals for All-Frequency Relighting”, ACM Transactions on Graphics, July 2004.
It is also known to combine multiple images to generate a light-field picture, which can then be presented in an interactive manner that allows for selective refocusing, relighting, and/or other effects. The above-referenced U.S. Utility Patent Applications provide several examples of mechanisms by which multiple images can be acquired, combined, stored, and used.
In general, light-field photography captures information about the direction of light as it arrives at a sensor within a data acquisition device such as a light-field camera. Such light-field data can be used to create representations of scenes that can be manipulated by a user. Subsequent to image capture, light-field processing can be used to generate images using the light-field data. Various types of light-field processing can be performed, including for example refocusing, aberration correction, 3D viewing, parallax shifting, changing the viewpoint, and the like. Many of these and other techniques are described in the above-referenced U.S. Utility Patent Applications. In addition, images from multiple capture devices can be combined to generate a “bullet-time” effect, wherein an apparent camera angle for a scene can be changed independently of the action taking place in the images themselves. This gives the impression that the camera is moving around a still image (or a moving image, such as one presented in slow motion). Bullet-time can be applied to still or moving images.
All of these effects and approaches require the collection of multiple images having different characteristics, such as different lighting positions and/or camera positions. Collection of images having different lighting characteristics must be done sequentially, as it is not generally possible to capture multiple images having different lighting characteristic at the same time; the subject can only be lit in one particular way at a given time. Existing techniques for collection of multiple images for image relighting and/or bullet-time involve specialized systems for implementing multi-camera and/or multi-flash imaging; these typically require expensive, cumbersome equipment, and significant effort to setup the lighting environment and subjects. These constraints make such advanced imaging techniques inaccessible to many ordinary consumers.
In addition, existing techniques fail to provide an easy way to configure devices (such as cameras, flashes, and the like) to communicate with one another in a manner that facilitates the capture of images.
In addition, existing techniques fail to provide an effective mechanism for assembling multiple images captured using multiple devices with one another for presentation to the user. Existing techniques also fail to provide an effective mechanism for providing interactive presentation of such assembled images.